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Augment human body and its environment to 
explore relations models between men and 
machines (digital interaction?) 


Gestural Control of Music 


From mice and sliders 
to virtual / augmented instruments 

and... 


Biophysical Control of Music 


...biosensing technologies 

(Rosenboom, Knapp, Tanaka, Nagashima, Miranda) 



Alvin Lucier 1965, Music For Solo Performer 




Long-term research outcome 


Implementation of low cost, open source tools (software and hardware) 
for biophysical-aided sound design in a real time environment: 


Re-distributable 
Customizable 
Easy to set up 
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The realization process needed to be fragmented into more specific and measurable steps. 
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Aesthetic principles 

for an experimental coupling between theatrical gesture and muscle sounds. 


• Authenticity of the performer's somatic interaction; 

• Natural responsiveness of the system; 

• Expressive immediacy and transparency of the 
mapping of muscle sounds to the performer's kinetic 
behaviour. 
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Methods 


• Aesthetic study of the acoustics of muscle sounds 

(implementation of an audio synthesis model for muscle sounds) 

• Development & design of the Xth Sense 

(a wearable biophysical sensor device) 

• Implementation of a Pd-based -rt DSP framework 

(to capture, analyse and design with muscle sounds) 

• Performance testing 

(to establish mapping and design definitions) 
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A synthesis model for muscle sounds 
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pd trigger 
pd params 


Disc rata summation synthesis 


MUSCLE SIMULATION 
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Human muscle sound synthesis model. 

Coded by Marco Donnarumma, based 
on discrete summation synthesis 
algorhythm included in 'Designing 
Sound' by Andy Farnell 

Edinburgh, 2010 












The Xth Sense / Sensor implementation 


a wearable, unobtrusive and 
extremely sensitive device 

(efficient rt capture <==> very low latency); 

using 


the most inexpensive hardware 

and accessible production methodology 


(to foster re-distribution and openness of the hardware). 


(Open Source) Biomedical computing? 


silicon case 


accelerometer 


silicon coniact 

membrane 



muscle vibration 


.1.^1 Vf. 
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CMASP schematic by Jorge Silva, Prism Lab, 2004 



















...and 

hacking 


Prototyping... 












The Xth Sense 


Wearable sensor device 




Prototype v.0.1 



The Xth Sense / Real time DSP modular framework 
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The Xth Sense / Real time DSP modular framework 
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Extracting useful data 


Mechanical Myography (MMG) 

RMS amplitude features extraction 

8 variables: 

6 discrete events, 1 continuous moving event and 

1 continuous exponential event. 

Mapping kinetic energy to control data 
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Performance testing: 
mapping and design definitions 


• biological sounds as major sonic source and control data; 


• no direct interaction of the performer with a computer 

(conceal the latter from the view of the public); 

• distinct, natural and non-linear interaction between kinetic energy 
and sonic outcome 

(instinctively controlled by the performer); 


• rich, specific and unconventional vocabulary of gesture/sound 
definitions 


(unambiguously interpreted by the audience); 
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... approach and perceive 

performer's body as a 

musical instrument and 

its kinetic energy as an 
exclusive sound 
generating force. 




Music for Flesh II 

Solo piece for augmented muscle sounds (Xth Sense Tech) 



Issues of interest and improvements 


• Excellence in the quality of the sonic matter 

(more accurate and ad hoc processing); 

• Deeper understanding of muscle sounds spectrum and dynamics 

(better control of multi-layered muscles); 

• Multiple sensors / Pattern gesture recognition 

(ANN, Neural Networks?); 

• Wireless system 

(dance, theater). 
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Further research contexts 


Development residency/ies 

(Inspace, UoE, School of Informatics, Edinburgh,...) 


Workshop/s 

(NK Berlin, Culture Lab Newcastle, Transfera Madrid, ...) 

Presentations / Concerts 

(Pd mini Con, ImagineCreate, LAC, ICMC,...) 

Seminary 

(HGG - Danish National Dance School) 
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Pictures Courtesy of 
the author, NASA, 
Mark Daniels 
(Artistic Director 
@Inspace), 

Chris Scott 
(chrisdonia. co .uk). 


Thanks to 
the Pure Data 
Community, Linux 
Audio.org, 

Dr. Martin Parker 
@UoE. 




[bang( 

[thanks_for_your_time] 
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Portfolio 

http://marcodonnarumma.com 

Research 

http://res.marcodonnarumma.com 


Blog 

http://www.thesaddj.com 



